Grounding Imperatives to Actions is Not Enough: A Challenge for Grounded NLU for Robots from Human-Human Data

We present a proposal for a Natural Language Understanding method for simple pick-and-place robots which maps utterances to different levels in an action hierarchy. The hierarchy is a graph containing both lower-level action and higher-level goal levels. This attempts to overcome the surprising lack of overt imperative verb forms in natural task-oriented dialogue, which we show to be the case statistically in a human-human corpus. This proposal shifts the task away from mapping utterances to either actions or goals exclusively, and instead allows flexible mapping to both actions and goals during the interaction. We also show how a continuous communicative grounding mechanism is vital for achieving fluid interaction and show how confirmations and repairs can refer to both the goal and action levels, and that reliance on these overt signals of understanding alone is inadequate for a natural model.